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Abstract 

The fundamental question to which most clients want and deserve an answer is, "Am I going 
to get better (as a result of counseling)?” Although meta-analyses provide strong evidence 
supporting the efficacy of counseling in general, if one wants to make probabilistic statements about 
individual client outcomes — rather than about the more generalized outcome of counseling — a 
different approach is needed. Using clients’ intake (pre-counseling) and post-counseling ratings of 
common problems derived from a multi-center database, an ordinal regression analysis was used to 
estimate the probabilities of four categories client improvement (or lack thereof) using six 
independent variables. Four of these variables were found to be significant predictors of outcome in 
a previous study; two were included as a result of discussions about the previous study. Five of the 
variables were based on information that a counselor meeting a client for the first time would (or 
could) reasonably have available. Our results show that previous experience as a client, readiness to 
change, level of symptomatic and interpersonal distress, pre-counseling clinical status, and the 
number of counseling sessions clients in which a client might be involved are significantly related to 
probabilities of outcome category membership. 
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Predicting Categories of Improvement Among Counseling Center Clients 

Counseling clients can be viewed as consumers who have the need and a right to know the 
benefits they will receive from counseling. Although clients may frame the question in various ways, 
we believe that the fundamental question to which most clients want and deserve an answer is, "Am I 
going to get better (as a result of counseling)?” The answer to this question is a categorical one. For 
example, as a result of their counseling, clients may deteriorate and show a negative change, remain 
unchanged, show small to moderate positive change, or show large and “clinically significant” or 
clinically meaningful change. In order to be able to answer the question of whether the client is going 
to get better, counselors need their own answers to questions regarding "What kinds of outcomes are 
likely with what types of clients?" For both clients and counselors, the answers to these questions 
must be probabilistic in nature, communicating the likelihood of improvement or the likelihoods of 
various categories of improvement or types of outcome. 

When considering counseling outcome, counselors are have been admonished to attend to 
characteristics of the client and the counselor, the type of counseling, and the nature of the outcome 
sought (e.g., Beutler & Clarkin, 1990; Kiesler, 1966; Paul, 1967). In this regard, counseling 
researchers have proceeded in their efforts by researching a few variables, some considered at only a 
few levels, and then analyzing the gathered data by postulating (at least implicitly) rudimentary 
models of the counseling effect. 

A reasonable indicator of how a particular client will do in counseling could be based on the 
outcomes of similar clients. But if counseling outcome varies as a function of client characteristics. 
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then there must be sufficient outcome data on each type of client to make a reliable statement about 
client improvement. 

One way to accomplish this is to aggregate the mean differences found across individual 
outcome studies and submit them to meta-analyses (e.g., see Smith, Glass & Miller, 1980). But 
while the effect size found in such meta-analyses usually provides strong justification that counseling 
is effective (which is important for the profession), the simple probabilistic statements deriving from 
the effect size provide a questionable basis for an individual client to decide whether or not to pursue 
counseling or for a counselor to assume that his or her work with a particular client will be effective. 

In this regard, Hummel and Lichtenberg (1999) noted that although it is not the primary 
purpose of meta-analyses to make probabilistic statements regarding a client’s likelihood of 
improvement as a result of counseling, doing so is not uncommon. Within reports on their studies, 
investigators frequently are found to present figures of overlapping normal distributions (one 
representing clients who received counseling and the other representing those in the control 
population). They then point out (by way of example) that a person at the mean of the treated 
population fell at the 75 th percentile of the control group — deducing that the probability is 0.75 of an 
individual randomly drawn from the treated population being above the mean of the control 
population. (Note: Given the symmetry of the normal distribution, if the mean of those treated falls 
at the 75 th percentile of the control population, then the mean of the control population must fall at 
the 25 th percentile of the treated population. Therefore, 75% of those treated must fall above the 
mean of the control population.) 

But it would be incorrect to assume that such a figure is synonymous with the probability 
that a particular client will show improvement, and in their paper, Hummel and Lichtenberg (1999) 
explored the limits of meta-analysis as a basis for this sort of justification of individual counseling 
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interventions. They concluded the while meta-analyses can (and usually do) provide strong 
justification for the profession, if one wants to make probabilistic statements about individual client 
outcomes — rather than about the more generalized outcome of counseling — one has to take a 
different approach. 

In the discussion section of their paper, Hummel and Lichtenberg noted that in principle there 
is no reason why client outcome data could not be aggregated across collection sites (e.g., counseling 
centers), rather than across studies, in order to accumulate sufficient outcome data on each type of 
client in order to make reliable statements about client improvement. They commented that if 
sites/centers would standardize descriptions of counselors, treatments, clients (and client problems), 
and outcomes, they could cooperate in development and maintenance of a large database of client 
outcomes and use these data to provide clients and counselors with accurate probabilities for various 
outcomes. 

Such a database exists, and at last year’s annual meeting of the AERA, Lichtenberg and 
Hummel (2000) reported on an analysis of the outcomes of 181 1 counseling clients seen at 38 
different U.S. college and university counseling centers around the country. Client intake information 
(e.g., age, gender, ethnicity, readiness for counseling, symptom severity, history of previous 
counseling, current medication) — the limited sort of information that counselors might have on which 
to base an answer to a client’s question regarding the likelihood of improvement — was used to 
predicted client outcome. The outcome variable in this study was a dichotomous variable (improved 
= a reduction in client symptoms; unimproved = lack of change or an increase in client symptoms). 

The first analysis in that study was a preliminary logistic regression entering each of the 
available independent (intake) variables simultaneously as a block. This analysis (similar in intent to 
running an overall MANOVA) was run in order to demonstrate to us that our intake variable set 
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related significantly to the probabilities of outcome category membership. The second analysis was 
a stepwise logistic regression, using the same variable set — with each variable entering the final 
equation on the basis of its unique contribution to the estimation of the client outcome category 
membership probabilities: ' 

The results of the analyses showed that previous experience as a client, readiness to change, 
and level of symptomatic and interpersonal distress were significant contribution to the estimation of 
the probabilities of outcome category membership. Specifically, clients who reported greater levels 
of distress and a greater readiness to change and who had not previously been in counseling were 
those most likely to improve in counseling (i.e., show an overall reduction in self-reported symptoms 
and distress). 

In the discussion that followed the presentation of the Lichtenberg and Hummel paper, 
several issues were raised. Specifically, it was suggested that outcome findings could be enhanced by 
considering (a) additional categories of improvement, (b) whether or not the clients entered 
counseling at a “clinical level” of distress, (c) the reliability of client change, and (d) the length of 
counseling (number of sessions) received by the client. 1 In light of these issues, the purpose of this 
study was to reanalyze the counseling center data previously presented by Lichtenberg and Hummel 
with the goal of building upon and further clarifying and enhancing the prediction of clients’ 
probabilities of various categories of counseling outcome. 



1 Technically, the length of counseling is not information a counselor can know at the onset of counseling. Within 
the context of this study, the number of sessions completed by the client is a conditional variable — that is, the 
outcome prediction made for a particular client is based on the intake variables conditional on the client’s completing 
some specified number of sessions. 
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Method 

Participants 

Participants in this study were the same 1811 counseling clients reported upon in the 
_previpusXichtenberg and Hummel study._The participants were a sub-sample of 4679 clients 
participating in a nationwide study of college and university counseling centers conducted by the 
Research Consortium of Counseling and Psychological Services in Higher Education. The clients 
whose data were analyzed were those clients on whom complete data were available at intake on the 
independent variables used in this study and on whom outcome data and number of counseling 
sessions completed also were available. 

Of our sample, one third were male. The majority (77%) of clients were Caucasian. The age 
of the clients varied from 16 to 60 (M = 23.01, SD = 5.4). Fifty percent (50%) of the clients 
previously had been in counseling, and 7% were currently on medication. Sixty-four percent (64%) 
of the clients had pre-counseling scores > 63 on the OQ-45 total score, placing them in the “clinical” 
category at intake. 

The counselors who saw the clients included practicum students, interns, and professional 
staff of the centers. The counselors (N=approx. 260) represented a variety of fields, although most 
were counseling psychologists. They represented diverse ethnic backgrounds, although most were 
Caucasian. The majority were female, although a significant minority of the counselors were male. 
Approximately half of the counselors were student trainees, the majority of whom were counseling 
center interns, although a significant minority were practicum students. 

The data for this study were provided by the Research Consortium of Counseling and 
Psychological Services in Higher Education, which was established in 1990 to further research efforts 
on the practices of college and university counseling centers and the concerns of their clients. The 



Predicting Improvement 8 



data analyzed were collected over the period of 1997 and 1998 and represent a subset of the data 
available from the consortium. Data routinely collected on the clients of the consortium member 
centers served as our data source. 

Instruments - — ______ _ — — — * — 

The instruments used in this study were the same ones reported in the Lichtenberg and 
Hummel (2000) study. They included the Outcome Questionnaire-45 (version 2) (OQ-45; Lambert, 
Hanse, Umpress, Lunnen, Okiishi, Burglingame, & Reisinger, 1 999) and the Stages of Change Scale 
(SCS; McConnaughy, Prochaska & Velcier, 1983). 

Outcome Ouestionnaire-45 (version 2) (OQ-45; Lambert, Hanse, Umpress, Lunnen, Okiishi, 
Burglingame, & Reisinger, 1 999). Pre- and post-counseling data were derived from the Outcome 
Ouestionnaire-45 . The OQ-45 is an instrument designed to measure client progress in therapy along 
three dimensions conceptualized by Lambert (1983) as important aspects of an client’s life: (a) 
subjective discomfort (i.e., how a person feels inside), (b) interpersonal relationships (i.e., how a 
person gets along with significant others), and (c) social role performance (i.e., how they manage in 
important life tasks such as work and school). 

The OQ-45 provides a total score and three individual domain scores. Each item on the 
questionnaire is scored on a five-point Likert scale (from 0-4), with some items reverse scored. The 
Symptom Distress fSDl score consists of 25 items and can vary from 0-100. The Interpersonal 
Relations (IR) score consists of 1 1 items and can vary from 0-44. The Social Role (SR) scale 
consists of 9 items and can vary from 0-36. Clinical cutoff scores have been established, with Total 
scores of 63 or greater, Symptom Distress scores of 36 or greater, Interpersonal Relation scores of 15 
or greater, and Social Role scores of 12 or greater reflecting clinical levels of distress. Lambert and his 
colleagues have found no gender or racial differences on the OQ-45, suggesting that the OQ-45 does 
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not over- or underpathologize any particular gender or racial group. Test-retest reliabilities for the 
OQ-45 vary from a low of .78 for the Symptom Distress scale to .84 for the OQ-45 Total score 
(N=157; the retest time interval was not stated in the instrument’s manual). Internal consistency 
coefficients (coefficient alpha) for the scales varied from .70 for the Social Role scale to .93 for the 
Total score (N=157). The concurrent validity of the OQ-45 has been estimated by correlating the 
scores of the instrument with the SCL-90-R (Derogatis, 1977). Notably, the OQ-45 Total score and 
the OQ-45 Symptom Distress score have been found to correlate highly with the General Severity 
Index (GSI) of the SCL-90-R (.78 and .82, respectively) for college counseling center clients. These 
results suggest considerable overlap between these indices of client symptomatic complaints. 

Stages of Change Scale (SCS; McConnaughy, Prochaska & Velcier, 1983). The Stages of 
Change Scale (SCS) is designed to assess clients’ readiness for change and their readiness to enter into 
and benefit from counseling. The scale assesses clients in terms of Prochaska’s stages of 
change — stages at which clients might enter and begin counseling. Each stage is measured by eight 
Likert-type items. The first stage. Precontemplation, characterizes clients that are not choosing to 
change themselves. The second stage. Contemplation , characterizes clients who are aware of 
problems and may wonder whether counseling would be helpful to them. The third stage. Decision 
Making , characterizes clients who have defined their problems and made a commitment to change. 
The fourth stage. Action, characterizes clients who are actively working on their problem(s) or 
concem(s). A final stage. Maintenance , characterizes clients who are at the point of consolidating 
any changes they may have made in counseling. The authors report adequate reliability and validity 
evidenced over numerous studies. (Note: Although Prochaska’s stages model includes five stages, 
the SCS data only included scale information for the Precontemplation, Contemplation, Action and 
Maintenance stages; absent is the Decision Making stage.) 
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Procedure 

Participating clients completed the OQ-45 and SCS prior to their initial intake appointment 
with their individual counselor. Although the OQ-45 was readministered throughout the course of 
counseling, only the clients’ initial OQ-45 and their final OQ-45, completed at termination, were 
used to assess counseling outcome. 

Using their intake (pre-counseling) scores on the OQ-45, clients were designated as either 
“clinical” or “non-clinical” at the onset of their counseling. In accordance with the OQ-45 manual, 
clients with OQ-45 total scores >63 were designated as “clinical.” 

A client’s outcome status, our dependent variable, was defined in terms of client change on 
the combined OQ-45 Total scale. Intake and post-counseling ratings on the OQ-45 were used to 
calculate a categorical index of improvement. The four ordered categories of client improvement (our 
dependent variable) were based on the OQ-45’s “reliable change index” (RCI) (Jacobson, Follette, & 
Revenstorf, 1984) as reported in the instrument’s manual (RCI =14). We used the RCI to construct 
an approximately 90% confidence interval on the true change score (observed change score ±14). 

Our four categories of improvement were: (1) 80 through 14 = some degree of reliable negative change 
(i.e., the client became worse), (2) 13.99 through -13.99 = no reliable change (i.e., the change could 
not be reliably distinquished from measurement error), (3) -14 through -33.99 = small to moderate 
reliable positive change (i.e., the client became better); and (4) -34 through —98 = large positive 
reliable [“clinically significant”] change (i.e., the client was much better). [NOTE: (a) The break 
point of -34 for large positive reliable change was determined by adding -14 to -20, where 20 points 
represents a large effect size obtained by multiplying .80 times the standard deviation of the OQ-45 
(SD - 25). Therefore, if a person scored -34, we could be about 90% confident that the person’s 
true change score was greater than 20 points. “Clinical significance” in the sense used above is a 
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function of the reliability and the magnitude of the change — regardless of the initial status (“clinical” 
vs. “non-clinical”) of the client, (b) A negative number indicates a decrease in client symptoms 
which is a positive change.] 

Analysis 

An ordinal logistic regression was used to estimate probabilities of improvement category 
membership as a function of the same four variables found to be significant predictors in the previous 
study’s model (a client’s intake scores on the OQ-45 social and interpersonal functioning scales and 
on the Contemplation subscale of the SCS and their history of previous counseling), together with 
the addition of length of counseling (number of sessions) completed by the client, and the client’s 
clinical status at intake (clinical/nonclinical). 

By (a) using a system of four outcome categories based on confidence intervals on the clients’ 
true change scores — a system that included a category of “clinically significant change” (effect size > 
.80), mid by including as predictors (b) the pre-counseling clinical status of the clients and (c) their 
length of counseling, our analysis addressed each of the specific concerns raised regarding the 
previous analysis of counseling outcomes. 

Results 

Table 1 summarizes the means and standard deviations for the six independent variables used 
in the analysis. 



Insert Table 1 about here 



Using the four categories defined above as the dependent measure and the four independent 
variables from the previous year’s study, an ordinal logistic regression was run. The prediction was 
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highly significant, x 2 (4) = 365.46, p too small to evaluate, and each of the individual predictors made 
a significant contribution. Adding the variable of the number of counseling sessions to the equation 
significantly improved the prediction, % 2 (1) = 1 1.27, p= .001. When the client’s pre-counseling 
clinical status was added next, it also made a significant contribution, x 2 (l) = 4.20, p = .041. The 
overall model based on our six independent variables yielded a chi-square of % 2 (6) = 380. 918, _p to 
small to evaluate. Table 2 summarizes the regression weights (coefficients) assigned to each variable 
in the prediction equation. Table 3 presents the frequency and proportion of clients in our sample 
that were categorized in each of the four outcome categories, as well as the proportion of clients 
estimated by our model to be in each of those categories. 

Insert Tables 2 and 3 about here 



Examples of outcome predictions derived from the ordinal logistic regression model are 
presented in the following series of figures — each depicting the probabilities of the various categories 
of counseling outcome as a function of different client values on the predictor variables. Figure 1 
displays the outcome category estimations for a client who would be classified as “clinical” (OQ-45 
Total > 63) at the onset of counseling, who had not previously been seen for counseling, and whose 
scores on the four continuous variables (symptom distress, interpersonal relations, contemplation 
about change in counseling, number of sessions) approximated the mean (or “typical client”) score 
for clients in our sample. Under such conditions, the client would have an approximately 46% 
probability of showing some reliable improvement: 31% (reliably better) + 14% (reliably much 
better) and a 10% probability of showing reliable deterioration. 
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Insert Figure 1 about here 



In Figure 2 we model a client has not previously been seen for counseling and whose 
symptom distress, interpersonal relations, and contemplation scales and number of counseling 
sessions completed are 1.5 standard deviations above the mean for our sample. Even without 
considering the OQ-45 Social Roles scale (which does not enter our model), the client’s combined 
means on the symptom distress and interpersonal relations scales designate the client “clinical” at 
intake. Such a case might exemplify an individual new to counseling, who is experiencing extreme 
(i.e., clinical levels of) symptomatic and interpersonal distress, is well aware of his problem and 
wondering whether counseling might be helpful, and who committed (i.e., completed) 4 sessions of 
counseling. For such a client, the model would estimate an 80% likelihood of reliable improvement: 
35% (reliably better) + 45% (reliably much better) and only a 1% probability of showing reliable 
deterioration. 



Insert Figure 2 about here 



In Figure 3 we model a client a client who previously has been seen for counseling, and whose 
scores on symptom distress, interpersonal relations, contemplation and number of sessions variables 
are set at 1.5 standard deviations below the mean of our sample. Such a case might exemplify a client 
previously for counseling who is experiencing relatively little symptomatic and interpersonal 
distress, is not really thinking about counseling, and who in fact is seen only for the intake session. 
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For such a client, the model would estimate only an 8% likelihood of reliable improvement and a 35% 
probability of deterioration. 



Insert Figure 3 about here 



The three figures depict the ordinal logistic regression model as implemented within an 
EXCEL spreadsheet. By selecting or entering different individual client values for each of the 
independent variables, one can model the effect of these different configurations of values on the 
estimated the probabilities of each type of client outcome. 

Discussion 

The results of our analysis show that previous experience as a client, readiness to change, 
level of symptomatic and interpersonal distress, pre-counseling clinical status, and the number of 
sessions to which clients commit themselves are significant estimators of the probability of outcome 
category membership. More specifically, the probability of clients’ membership in each of four 
counseling outcome categories ( reliable negative change/client became worse , no statistically reliable 
change , small to moderate reliable positive change : and large, reliable positive [“clinically significant”] 
change ') could be reliably estimated based on their standing on several routinely collected intake 
variables, and conditional on the client completing a certain specified the number of counseling 
sessions. 

The result of our previous study (Lichtenberg & Hummel, 2000) suggest that clients who 
report greater levels of social and symptomatic distress and a greater readiness to change and who 
have not previously been in counseling are those most likely to improve in counseling (i.e., show a 
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greater overall reduction in self-reported symptoms and distress). Our current analysis shows that 
the predictive power of these intake variables can be enhanced by adding to them a client’s pre- 
counseling clinical status on the OQ-45 (“clinical” vs. “non-clinical”) and the number of sessions the 
client may complete. 

These results build upon as well as clarify and enhance the estimates of clients’ probabilities 
of various categories of counseling outcome over those presented previously by Lichtenberg and 
Hummel (2000). Beyond merely estimating the probability of whether or not an individual client is 
likely to improve as a result of counseling (i.e., a simple reduction in client-reported symptoms), this 
reanalysis allows for the estimation of the likelihood of client deterioration, non-improvement, small 
to moderate improvement, and large, “clinically significant” improvement. And the explanatory 
power of the ordinal logistic regression equation was found to be enhanced by considering a client’s 
pre-counseling level of symptomology (“clinical” vs. “non-clinical”) and the amount of counseling 
(number of sessions) that a client might complete. 

As data are collected within and across counseling settings, providers of counseling services 
will learn more about what interventions work for what kinds of clients under naturalistic counseling 
conditions and will be better able to provide client consumers with the information they need before 
deciding whether or not to enter into counseling — information that we believe needs to include a 
response to the question, “Am I going to get better?” A response to this question necessarily must 
be probabilistic. Counseling centers, like other behavioral healthcare settings, need to be able to meet 
the primary goals of health care, namely, to help the client when possible and to do no harm (Lyons, 
Howard, O’Mahoney & Lish, 1997; Ogles, Lambert & Masters, 1996; Sederer, Dickey & Hermann, 
1996). Meeting these goals requires being able to establish and communicate realistic probabilities 
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for various counseling outcomes for the individual clients that are seen by their counseling staff so 
that these clients can make informed decisions about entering into counseling. 
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Table 1 

Means and Standard Deviations for the Six Independent Variables (n=181 I s ) 



Variable 


M 


SD 


Previous Counseling (yes/no) 


.50 


.50 


OQ-45 Symptom Distress 


41.50 


16.48 


OQ-45 Interpersonal Relations 


16.34 


6.94 


SCS Contemplation 


34.45 


3.76 


# Sessions 


4.05 


3.83 


OQ-45 > 63 (“clinical” vs. “non-clinical”) 


.64 


.48 



Note: The mean for Previous Counseling is the proportion of clients responding “yes” to the item. 
The mean for OQ-45 > 63 is the proportion of clients classified as “clinical” on the basis of their 
OQ-45 Total Score at intake. 
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Table 2 

Logistic Regression Table 



Predictor 


Coefficient 


St. Dev. 


Z 


P 


Constant (1) 


1.6193 


.4291 


3.7700 


.0000 


Constant (2) 


4.6095 


.4414 


10.4400 


.0000 


Constant (3) 


6.2201 


.4516 


13.7700 


.0000 


Previous Counseling 


.0305 


.0905 


3.3700 


.0010 


Symptom Distress 


-.0375 


.0045 


-8.3600 


.0000 


Interpersonal Relations 


-.0216 


.0082 


-2.6400 


.0080 


Contemplation 


-.0600 


.0124 


-4.8400 


.0000 


# Sessions 


-.0407 


.0117 


-3.4700 


.0010 


OQ-45 > 63 


-.3189 


.1550 


-2.0600 


.0400 
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Table 3 

Proportions of Clients in Each of the Outcome Categories 





Empirical n 


Empirical 


Estimated Proportions 


Outcome Cateeorv 


in Each Outcome 
Category 


Proportions 


Based on the 
Prediction Equation 1 


Worse 


176 


.097 


.071 


Can’t Tell 


898 


.496 


.532 


Reliably Better 


474 


.262 


.281 


Reliably Much Better 


263 


.145 


.116 



1 Estimated roportions based on the prediction equation were derived by using the mean value for 
each of the predictor/independent variables in the prediction equation. 
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Figure 2 
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